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The HIV-1 BED incidence assay was adopted in China in 2005 for HIV-1 incidence surveillance. A 
proficiency testing (PT) program was established in 2006 to provide quality assurance services. The BED PT 
program consisted of two components, an international program provided by the U.S. Centers for Disease 
Control and Prevention from 2006 and a domestic program started by the National HIV/HCV Reference 
Laboratory in 2011. Each PT panel consisted of eight coded specimens distributed to participating 
laboratories semi-annually, and testing results were collected and analyzed. The number of participating 
laboratories increased progressively from 2006 to 2012. The Chinese HIV-1 incidence laboratory network 
performed satisfactorily both in international and domestic PT programs. We also demonstrated that the 
BED assay was highly reproducible among participating laboratories. Our success and lessons learned can be 
readily replicated in other countries or regions contemplating the establishment of a PT program for 
assay-based HIV incidence estimation. 



HIV incidence determination is a vital component of monitoring and understanding HIV transmission 
dynamics in high-risk populations and measuring prevention effectiveness 1 . Traditionally, HIV incidence 
estimation has been carried out using prospective cohort studies that pose many logistical and cost 
challenges 2,3 . With the recent development of laboratory techniques able to distinguish between recent and 
long-term HIV-1 infections, the use of cross-sectional surveys for HIV-1 incidence estimation has become 
possible 4 . Among the many laboratory-based incidence assays developed, the most evaluated assay is the 
immunoglobulin G (IgG)-capture BED- enzyme immunoassay (BED-CEIA) based on measurement of the 
proportion of HIV-1 -specific IgG to total IgG after seroconversion 5 7 . Licensing and manufacturing of commer- 
cial kits with consistent quality began in 2005 and this assay is currently used globally for population 
surveillance 1,810 . 

Beginning in 2005, the Chinese National HIV/HCV Reference Laboratory and several other provincial laborat- 
ories adopted the BED incidence assay for HIV- 1 incidence surveillance. Collectively, they demonstrated high 
inter-laboratory reproducibility and yielded HIV-1 incidence estimates similar to those obtained by prospective 
cohort studies in China 1,11 " 14 . As the number of laboratories performing incidence assays rapidly increased, the 
need to monitor overall and individual performance increased. The National HIV/HCV Reference Laboratory 
has an extensive quality assurance (QA) and quality control (QC) system for HIV/ AIDS testing, including 
laboratory staff training, supervision, management, accreditation, proficiency testing (PT), and diagnostic kit 
evaluation for serologic, immunologic, and molecular assays 15,16 . 

PT is a key component of external quality assurance, which utilizes inter-laboratory comparison to determine 
the ability of laboratories to obtain accurate results and has been shown to be effective in monitoring and 
improving HIV-related laboratory testing quality 17 23 . The goals of PT for HIV incidence testing are to assess 
the performance of participating laboratories and identify issues for corrective action and technical assistance. 
The National HIV/HCV Reference Laboratory organized provincial BED laboratories to voluntarily participate 
in the semiannual international BED PT program run by the U.S. Centers for Disease Control and Prevention 
(CDC) in 2006 and subsequently developed a domestic BED PT program in 2011. In this report, we summarize 
performance characteristics of the international and domestic BED PT programs. 
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Figure 1 | Participation and performance of laboratories of China in the international BED PT program over 13 rounds (a) and in the domestic BED 
PT program over 3 rounds (b). 



Results 

Participation and overall rating. In addition to the National HIV/ 
HCV Reference Laboratory, 7 other Chinese laboratories partici- 
pated in the international BED PT program provided by the U.S. 
CDC in 2006; this number increased to 19 laboratories in 2012 
(Fig. la). In round 10, one laboratory participated in the PT pro- 
gram at a late stage and shared one panel with another laboratory 
which resulted in more reports sent than the number of panels 
received. Overall, there were 182 results, all with valid QC and 
correct recency classifications, and thus all Chinese laboratories 
received a "pass" rating in the international program over 13 rounds. 

For the domestic PT program, there were 31 and 32 participating 
laboratories in 2011 and 2012, respectively (Fig. lb). Ninety (95.7%) 
of the 94 returned results achieved a perfect score over 3 domestic PT 
rounds. Three (9.7%) laboratories in the initial round and one (3.1%) 
in the third round failed to attain a pass rating. Three of these four 
laboratories with failed rating were new PT participants. 

PT data analysis. A total of 104 specimens were distributed over 13 
international PT rounds, and 24 specimens were distributed over 3 
domestic PT rounds. Overall, the concordance plots between 
participant normalized optical density (ODn) and expected ODn 
results are shown separately for international and domestic PT 



(Fig. 2a) and by kit manufacturers (Fig. 2b and 2c). The agree- 
ments between obtained and expected results were high with 
correlation coefficients (R 2 ) ranging from 0.942 to 0.998. Bland- 
Altman plots of the mean differences were all close to zero ( — 0.029 
and 0.199 for international and domestic PT, respectively), and the 
95% limits of agreement include 92.3% (96/104) and 91.7% (22/24) of 
all difference data from international (Fig. 2d) and domestic (Fig. 2e) 
PT, respectively. Similar results were also obtained between Calypte 
and Sedia kits (result not shown). Results in the lower left quadrant 
(Fig. 2a, 2b and 2c) represented incident specimens and those in the 
upper right quadrant represented long-term infections. There was 
one specimen in round 12 of the international program and ano- 
ther in round 1 of the domestic program that yielded discordant 
classification (lower right and upper left). Because ODn values of 
these specimens were close to the 0.8 cutoff for recency determina- 
tion, the specimens are expected to fluctuate around the cutoff and 
laboratory performance was considered satisfactory. 

QC analysis. The BED assay kit contains 4 controls including a 
calibrator (CAL) to validate each plate run and normalize the 
optimal density of specimens in relation to CAL optical density 
(OD) which reduces inter-run and inter-laboratory variability. 
There were 364 plate assays (210 and 154 by Calypte and Sedia 
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Figure 2 | Concordance of mean obtained results with expected results for 13 international BED PT rounds (n = 104) and 3 domestic BED PT rounds 
(n = 24). Regression analysis: correlation of expected ODn results and Mean ODn results. The best-fit regression line is shown for each PT program. 
Vertical and horizontal arrows indicate 0.8 cutoffs. Mean ODn results vs. expected ODn results for all participants (a); participants using kits from 
different manufacturers during international PT (88 Calypte kits, 40 Sedia kits) (b); participants using kits from different manufacturers during domestic 
PT (Calypte kits, n = 8; Sedia kits, n = 24) (c). Bland-Altman analysis: correlation of expected ODn results and Mean ODn results for international PT 
program (d) and domestic PT program (e). 



kits, respectively) conducted in 13 rounds of the international PT 
program. There were 183 plate assays (26 and 157 by Calypte and 
Sedia kits, respectively) in the 3 domestic PT rounds. The 
performance characteristics of these runs are shown in Table 1. In 
the international PT, coefficients of variation (CVs) of the mean OD 
values of low positive control (LPC) and high positive control (HPC) 
were between 20.2% and 23.6% and were reduced to 6.3 to 7.1% after 
normalization, which illustrates a reduction in variability. The ODn 
readings obtained from Calypte and Sedia kits were very close. For 
instance, the mean ODn of LPC was 0.611 (CV 7.1%) and 0.594 (CV 
6.6%) for Calypte and Sedia, respectively. Similar narrow ODn 
ranges and CVs were observed in LPC and HPC in the domestic 
program. Negative control (NC) CV was high, as anticipated, due 
to their low OD readings. 

Laboratory reproducibility. In order to assess intra-laboratory 
reproducibility, overall concordance of initial ODn results with 
confirmatory ODn results were compared by regression analysis in 
Fig. 3a and 3b and by Bland-Altman analysis in Fig. 3c and 3d for 
international and domestic PT, respectively. The R 2 of the linear 
regression lines for international and domestic BED PT were 0.942 
and 0.905, respectively; The mean differences were all close to zero 
(0.015 for international PT and —0.003 for domestic PT), and the 
95% limits of agreement include 94.0% (739/786) and 94.2% (425/ 
45 1 ) of all difference data reported by international and domestic PT, 



respectively. Four of 451 results, as depicted in open circles, showed 
considerable deviation (Fig. 3b and 3d). 

The presence of 2 identical specimens in each international PT 
round was used to provide another insight into assay reproducibility 
within and between laboratories. The specimen used for the repro- 
ducibility check in each round was different and thus their ODn 
value changed from round to round. Fig. 4 shows data on duplicated 
specimens in Calypte and Sedia kits (Fig. 4a), individual Calypte 
kit (Fig. 4b), and Sedia kit (Fig. 4c). The mean ODn values of the 
duplicate specimens were very close to each other in each round, 
indicating high reproducibility. The average CV of duplicates was 
1 1.4%, and mean CV of Calypte and Sedia kits was 12.3% and 10.0%, 
respectively. 

Cause of laboratory error and corrective actions. All Chinese 
laboratories that participated in 13 rounds of the international PT 
program from 2006 to 2012 received "pass" ratings. In the domestic 
program, 4 reports of the 94 returned reports failed over 3 rounds 
(Table 2). There was one misclassification report in round 1, but 
because a corrective action report was not returned, the cause is 
unknown. A kit reagent problem in round 1 was not resolved 
because the laboratory did not have an additional kit to repeat the 
testing. An invalid QC result in round 3 was due to a transcriptional 
error when a technician incorrectly transferred data from the 
laboratory form to the PT report form. The remaining 2 results 
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with invalid QC were produced by 2 laboratories newly partici- 
pating in PT. 

Discussion 

To understand the expanding HIV epidemic, the Chinese govern- 
ment took steps to scale up HIV screening to identify infected per- 
sons for early treatment in order to reduce the mortality and 
transmission 24 " 26 . To better understand the trajectory of the HIV-1 
epidemic in most-at-risk populations, China adopted the laboratory- 
based BED incidence assay initially developed at the U.S. CDC in 
2002 s . Several incidence measurements were carried out by both BED 
assay and prospective cohort studies in China and yielded compar- 
able results 1,12 " 14 . With vigorous laboratory training and quality mon- 
itoring, the BED assay platform was quickly expanded to include 32 
provincial laboratories 16 . In 2011, the China CDC formulated a 
national incidence program to use provincial serologic confirmatory 
laboratories to conduct its annual national HIV incidence determina- 
tion 27 and designated the National HIV/HCV Reference Laboratory 
as the principle laboratory to coordinate and monitor overall per- 
formance and quality assurance. The use of multiple provincial ser- 
ologic confirmatory laboratories in China is in sharp contrast to the 
use of a centralized BED-based incidence laboratory in the U.S.. This 
de-centralized operation mode necessitates the needs of timely coor- 
dination, high quality reagent kits, uniform performance across vari- 
ous provincial laboratories and stringent quality monitoring. 

In 2006, the National HIV/HCV Reference Laboratory organized 
7 provincial laboratories to participate in the U.S. CDC's inter- 
national BED PT program. To reduce the international transporta- 
tion costs and make distribution more uniform, the PT panels were 
collectively shipped from the U.S. CDC to the National HIV/HCV 
Reference Laboratory. By 2012, the number of participating laborat- 
ories of this international program gradually increased to 19. China 
was then a major participant country in the U.S. CDC's BED PT 



program, constituting 45% of total program laboratories. Antici- 
pating the need of quality monitoring of the annual national HIV 
incidence survey, the National HIV/HCV Reference Laboratory took 
another major step in 2011 and established a domestic PT program. 
This domestic program was well accepted. There were 32 particip- 
ating laboratories in 2012 representing a 68% increase over the num- 
ber of Chinese laboratories in the international program. 

As reported by Dobbs et al 2s , it was critical to demonstrate the 
quality of commercial BED assay kits and the consistent quality 
performance across laboratories and operators. Our data from both 
international and domestic programs indeed demonstrated high cor- 
relations between participant and expected results (Fig. 2), outstand- 
ing agreement between initial and confirmatory tests (Fig. 3), and 
very low CV of duplicate specimen results (Fig. 4). In 3 rounds of the 
domestic program, 95.7% of reports correctly identified the recency 
status of all PT specimens. The R 2 , slopes and intercepts of the best- fit 
regression lines shown in Figs. 2b and 2c and the controls summar- 
ized in Table 1 illustrate minimal difference in performance between 
the two kit manufacturers. As expected, laboratory staff who parti- 
cipated in both the international and domestic PT performed better 
than those who participated only in the domestic program. 
Laboratories with invalid results were targeted for more extensive 
training and corrective remedies were distributed through a nation- 
wide laboratory newsletter. 

However, we also found the shortcoming of the domestic BED PT 
program from the summarized results. All the results of regression 
analysis over the domestic PT program were slightly inferior to the 
international PT. The probable reasons include inexperience of staff 
of some new participating laboratories, the challenges faced in stor- 
age and administration of specimens in some resource-limited set- 
tings. Additionally, the lack of duplicate specimens in each domestic 
PT round resulted in simplification of the approach for evaluating 
the mtra-laboratory reproducibility, only through assessing the con- 
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Figure 3 | Overall concordance from all participating laboratories between the initial and confirmatory runs for 786 results during international 
PT rounds and 451 results during domestic PT rounds. Regression analysis (aandb) and Bland-Altman analysis (candd) for international and domestic 
PT program, respectively. For each PT program, the best-fit regression line is shown with statistics in the upper left (a and b). Four results with 
large different initial and confirmatory ODn are indicated by open circles (b and d). 



cordance between initial and confirmatory results. As the domestic 
BED PT program is in the initial stage, more improvements will be 
achieved with its development. 

Our successful coordination and monitoring of the quality of 
laboratory-based HIV-1 incidence testing in a large national HIV 
testing network in China is highly encouraging. Our findings illus- 
trate the high quality of reagents and reproducibility of operators. 
Our success could be replicated by other countries or regional pro- 
grams. A new limiting-antigen avidity assay was recently reported to 
provide more accurate HIV-1 incidence estimation than the BED 
assay 29 . We are currently evaluating this new assay and will launch a 
domestic program based on the lessons learned from our current 
BED PT experience. 



International and domestic BED PT programs were successfully 
used to monitor the quality of laboratory-based incidence estimation 
and better understand the causes of testing errors. Recently, several 
laboratory-based methods have been devised to identify HIV-1 
incident infections 29 34 . As the usefulness of these assays is demon- 
strated and they are adopted, quality monitoring programs such as 
PT will need to be established. Our experience with the HIV- 1 BED 
incidence assay, first using an international program and subse- 
quently developing a domestic program, will guide the quick estab- 
lishment of PT programs for other assay platforms to strengthen the 
Chinese HIV laboratory network. Other countries or regions in the 
world can also learn from this experience to establish appropriate 
local PT programs for assay-based HIV incidence estimation. 
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Figure 4 | Performance characteristics of duplicated specimens in the international PT program rounds, with black and white bars representing the 
specimen replicates. Mean ODn with standard deviation (SD) of all participating laboratories (a), laboratories using Calypte kits (b), and 
laboratories using Sedia kits (c). Sedia kits became available in 2010. In round 13 of the international BED PT program and round 3 of the domestic BED 
PT program, all participating laboratories used Sedia kits except one. The results from that laboratory in these two rounds were not included. 



Methods 

PT panel specimens and performance reporting. The international PT program was 
provided by the U.S. CDC. The heat inactivated (56°C, 30 minutes) serum specimens 
(n = 8) chosen for each PT round were derived from a repository of 25 undiluted 
HIV-1 positive bulk volume donor plasma samples 28 . Their OD values and recency 
status were determined by HIV- 1 BED incidence assay in the U.S. CDC laboratory in 
Atlanta, Georgia. Specimens were aliquoted 100 ul per tube and stored at — 70°C 
prior to distribution to international participating laboratories. In addition, two of the 
eight specimens in each panel were duplicates to allow for intra-laboratory 



Table 2 | Laboratory failures and associated reasons during the 
domestic PT program d 



Cause 

Invalid QC 
Transcription error 
Misclassified specimen 
Kit reagent problem 



Number of occurrences 
Round 1 Round 2 Round 3 Total 



2 e 

0 
1 

}<• 



0 
0 
0 
0 



l f 

0 
0 



d The numbers of participating laboratories in rounds 1 , 2 and 3 were 3 1 , 32 and 32 respectively. 
e one laboratory in round 1 and f a different laboratory in round 3 had two errors each. 



reproducibility analysis. PT panels for all Chinese participating laboratories were sent 
in one shipment in each PT round with dry ice from the U.S. CDC to the National 
HIV/HCV Reference Laboratory. International transportation and customs 
clearance typically took 3-5 weeks. Upon receipt at the National HIV/HCV Reference 
Laboratory, PT panels were repackaged with ice packs and distributed to provincial 
laboratories. Domestic express mail usually took 2-3 days. Each participating 
laboratory was requested to report test results to the National HIV/HCV Reference 
Laboratory, who then transferred results to the U.S. CDC. In reverse manner, the U.S. 
CDC returned PT results to the National HIV/HCV Reference Laboratory who then 
distributed results to individual provinces. 

For the domestic program, PT specimens were derived from a repository of 40 
HIV-1 positive specimens purchased from Kinghawk Pharmaceutical Co., Ltd 
(Beijing, China). Eight specimens with vaiying OD values representing recent or 
long-term infection were randomly chosen from this repository for each domestic PT 
round and their OD values were verified prior to distribution. Aliquoted specimens 
(100 ul) were stored at -70°C at the National HIV/HCV Reference Laboratory. The 
number of specimens with recent or long-term classification varied from round to 
round and there were no duplicate specimens as in the international PT program. PT 
specimens were packaged and shipped by express mail service with ice packs. 
Participants were instructed to store the specimens at — 20' C or below prior to testing 
and not to freeze-thaw them repeatedly. They were also instructed to return results 
within 10 working days after receipt of specimens. 

For both the domestic and international programs, a cover letter and Microsoft 
Excel data report form were e-mailed to each participant. The report form contained 
the assay kit name, lot number, expiration date, technician name, assay date, QC and 
specimen OD and ODn values, and result interpretation for recent or long-term 
status. Laboratory directors were asked to verify data and sign off on the report. The 
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National HIV/HCV Reference Laboratory collected and analyzed results, and then 
summarized overall findings with the names of the laboratories staying anonymous. 
Each laboratory director was informed of his/her laboratory performance. The 
summary report was published in the Laboratory Network Newsletter, a China CDC 
periodical. 

BED CEIA assay and incidence classification. All specimens in the PT panel were 
tested by participating laboratories using BED HIV-1 Incidence EIA kits purchased 
from either Calypte Biomedical Corporation (Portland, OR) or Sedia Biosciences 
Corporation (Portland, OR). Kit performance comparisons were conducted for each 
round except for round 13 of the international program and round 3 of the domestic 
program in which only one laboratory each used Calypte kit and thus their results 
were not included in the analysis of Sedia performance. The assay procedure and 
testing rationale had been previously described 5 - 6,35 . Briefly, all specimens were first 
screened in singlet. A kit CAL, NC, LPC and HPC were run in triplicate on each EIA 
plate and median values were obtained. Specimen OD values were divided or 
normalized by median CAL OD to yield ODn. As required by both kits, specimens 
with initial screening results of ODn 1.2 or less were tested again in triplicate in the 
confirmatory run. Specimens in the confirmatory run with ODn ^ 0.8 were classified 
as recent infection and those with ODn > 0.8 as long-term. 

Data analysis and corrective actions. For each round, the National HIV/HCV 
Reference Laboratory collected information and reports from participating 
laboratories. The data from all 13 international and 3 domestic rounds was analyzed 
in aggregate. Overall reported results were compared to the panel's expected ODn 
values and recent/long-term classification by regression analysis and Bland- Altman 
analysis. Additional parameters measured include inter-laboratory comparison, 
inter-assay reproducibility of specimens retested by confirmatory testing, and intra- 
assay reproducibility of duplicate specimens. 

To pass a PT round, the OD and ODn values of all QC materials must fall within 
the established ranges shown in the kit inserts and the final classification of each of the 
panel specimen should be in complete agreement with expected final result inter- 
pretation established by the U.S. CDC in the international PT or by the National HIV/ 
HCV Reference Laboratory in the domestic PT program. A single invalid result in 
either QC materials or recency classification resulted in laboratory failure. If an error 
was found, the participating laboratory was notified and required to submit a cor- 
rective action form within two weeks to identify the source of the error. If requested, 
the National HIV/HCV Reference Laboratory assisted participating laboratories in 
identification of errors. 

The summary report for each BED PT round was published in the Laboratory 
Network Newsletter as mentioned, allowing each participating laboratory to compare 
their results to the expected results. 

Approval of the study. This study was approved by The Academic Committee of The 
National Center for AIDS/STD Control and Prevention of China CDC and all 
experiments were carried out in accordance with the approved guidelines. All the PT 
specimens were from commercial panels (SeraCare Life Sciences, Inc. and Kinghawk 
Pharmaceutical Co., Ltd for international and domestic PT program, respectively), so 
there existed no issues about ethics and informed consent. 
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